Design of Virtual 3D Instruments for Musical Interaction

An environment for designing virtual instruments with 3D geometry has been prototyped and applied to realtime sound control and design. It was implemented by extending a realtime, visual programming language called Max/FTS, running on an SGI Onyx, with software objects to interface CyberGloves and Polhemus sensors and to compute human movement and virtual object features. Virtual input devices with behaviours of a rubber balloon and sheet were designed for the control of sound spatialization and timbre parameters. Informal evaluation showed that a sonification inspired by the physical world appears natural and effective. More research is required for a natural sonification of virtual input device features such as shape, taking into account possible co-articulation of these features. While both hands can be used for manipulation, left-hand-only interaction with a virtual instrument may be a useful replacement for and extension of the standard music synthesizer keyboard modulation wheel. More research is needed to identify and apply manipulation pragmatics and movement features, and to investigate how they are co-articulated, in the mapping of virtual object parameters.